Biostats Lecture 5: Statistical Hypothesis Testing

class: center, top, .title-slide, title-slide

# Biostats Lecture 5: Statistical Hypothesis Testing
## Public Health 783
### Ralph Trane
### University of Wisconsin–Madison 
### Fall 2019

---

# Recap

Random variables

Distributions

Estimators

Estimators are random variables! (for example, the average is a random variable)

---
layout: true

# Statistical Hypothesis Testing

---

**Scenario**:

We've been playing a simple game. Everytime you roll a six, I pay you a dollar. Everytime I roll a six, you pay me a dollar.

I've had crazy good luck, and by the end of the day won a lot of money from you.

You accuse me of cheating, and demand to test the dice I've been using!

I agree to let you test them, but ONLY if you do it in a sound, statistical manner. How to go about that?

You decide to roll the dice `\(12\)` times each, for a total of `\(36\)` rolls. You assume they'll all behave the same, so the probability of rolling a six is the same for all three dice.

---

**Setup**:

Let `\(X_1, X_2, ..., X_{36}\)` be the outcomes of the thirty "trials". Each trial consists of rolling a die, and check if it's a six or not. If it's a six, we'll call it a success, if not we'll call it a failure. I.e. `\(X_i \sim\)`
--
 `\(\text{Bernoulli}(p)\)`.

*IF* the dice are fair, `\(P(X_i = 1) = 1/6\)` for all `\(i = 1,2,...,36\)`.

*IF* the dice are fair, we would expect to roll a `\(6\)` close to `\(\frac{1}{6}\cdot 36 = 6\)` times, i.e. about `\(5\)` of the `\(X\)`'s should be `\(1\)`'s.

What would cause you to reject the idea that the dice are fair?

If we see way more than `\(6\)` sixes. What would be "way more"? `\(7\)`? `\(8\)`? `\(17\)`?

---

In terms of probabilities: what is the *probability* of observing at least `\(10\)` sixes *IF* the `\(P(X_i = 1) = \frac{1}{6}\)`?

* if the probability is small, `\(10\)` is a lot of sixes
* if the probability is large, `\(10\)` is a reasonable number of sixes

First, introduce the random variable `\(Y =\)` number of sixes `\(= X_1 + X_2 + ... X_{36}\)`. The probability of observing more than `\(10\)` sixes is `\(P(Y \ge 10)\)`. To find this, we need the distribution of `\(Y\)`, which is 
--
 `\(\text{Binomial}(36, p)\)`, where `\(p\)` is the probability of rolling a six.

---

*IF* the dice are fair, `\(p = \frac{1}{6}\)`. So *IF* the dice are fair, the distribution of `\(Y\)` looks like this:

<center>
<div id="htmlwidget-24133c208c936b1de966" style="width:600px;height:450px;" class="plotly html-widget"></div>
<script type="application/json" data-for="htmlwidget-24133c208c936b1de966">{"x":{"data":[{"orientation":"v","width":[1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1],"base":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],"x":[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36],"y":[0.0014108314823815,0.0101579866731468,0.0355529533560137,0.0805866942736309,0.132968045551491,0.170199098305909,0.175872401582772,0.150747772785233,0.109292135269294,0.0680039952786718,0.0367221574504828,0.0173595653402283,0.0072331522250951,0.00267070236003511,0.000877516489725824,0.000257404836986242,6.75687697088885e-05,1.58985340491502e-05,3.35635718815394e-06,6.3594136196601e-07,1.08110031534221e-07,1.64739095671194e-08,2.24644221369811e-09,2.73479921667595e-10,2.96269915139895e-11,2.84419118534297e-12,2.40662331067484e-13,1.78268393383322e-14,1.14601110032136e-15,6.32281986384201e-17,2.95064926979291e-18,1.14218681411339e-19,3.56933379410432e-21,8.65293040994988e-23,1.52698771940293e-24,1.74512882217479e-26,9.69516012319318e-29],"text":["x: 0 Probability: 0.00141","x: 1 Probability: 0.01016","x: 2 Probability: 0.03555","x: 3 Probability: 0.08059","x: 4 Probability: 0.13297","x: 5 Probability: 0.17020","x: 6 Probability: 0.17587","x: 7 Probability: 0.15075","x: 8 Probability: 0.10929","x: 9 Probability: 0.06800","x: 10 Probability: 0.03672","x: 11 Probability: 0.01736","x: 12 Probability: 0.00723","x: 13 Probability: 0.00267","x: 14 Probability: 0.00088","x: 15 Probability: 0.00026","x: 16 Probability: 0.00007","x: 17 Probability: 0.00002","x: 18 Probability: 0.00000","x: 19 Probability: 0.00000","x: 20 Probability: 0.00000","x: 21 Probability: 0.00000","x: 22 Probability: 0.00000","x: 23 Probability: 0.00000","x: 24 Probability: 0.00000","x: 25 Probability: 0.00000","x: 26 Probability: 0.00000","x: 27 Probability: 0.00000","x: 28 Probability: 0.00000","x: 29 Probability: 0.00000","x: 30 Probability: 0.00000","x: 31 Probability: 0.00000","x: 32 Probability: 0.00000","x: 33 Probability: 0.00000","x: 34 Probability: 0.00000","x: 35 Probability: 0.00000","x: 36 Probability: 0.00000"],"type":"bar","marker":{"autocolorscale":false,"color":"rgba(89,89,89,1)","line":{"width":1.88976377952756,"color":"transparent"}},"showlegend":false,"xaxis":"x","yaxis":"y","hoverinfo":"text","frame":null}],"layout":{"margin":{"t":40.8401826484018,"r":7.30593607305936,"b":37.2602739726027,"l":48.9497716894977},"plot_bgcolor":"rgba(255,255,255,1)","paper_bgcolor":"rgba(255,255,255,1)","font":{"color":"rgba(0,0,0,1)","family":"","size":14.6118721461187},"title":{"text":"Binomial(n = 36, p = 1/6)","font":{"color":"rgba(0,0,0,1)","family":"","size":17.5342465753425},"x":0,"xref":"paper"},"xaxis":{"domain":[0,1],"automargin":true,"type":"linear","autorange":false,"range":[-2.35,38.35],"tickmode":"array","ticktext":["0","10","20","30"],"tickvals":[0,10,20,30],"categoryorder":"array","categoryarray":["0","10","20","30"],"nticks":null,"ticks":"outside","tickcolor":"rgba(51,51,51,1)","ticklen":3.65296803652968,"tickwidth":0.66417600664176,"showticklabels":true,"tickfont":{"color":"rgba(77,77,77,1)","family":"","size":11.689497716895},"tickangle":-0,"showline":false,"linecolor":null,"linewidth":0,"showgrid":true,"gridcolor":"rgba(235,235,235,1)","gridwidth":0.66417600664176,"zeroline":false,"anchor":"y","title":{"text":"Value of Y","font":{"color":"rgba(0,0,0,1)","family":"","size":14.6118721461187}},"hoverformat":".2f"},"yaxis":{"domain":[0,1],"automargin":true,"type":"linear","autorange":false,"range":[0,0.193459641741049],"tickmode":"array","ticktext":["0.00","0.05","0.10","0.15"],"tickvals":[0,0.05,0.1,0.15],"categoryorder":"array","categoryarray":["0.00","0.05","0.10","0.15"],"nticks":null,"ticks":"outside","tickcolor":"rgba(51,51,51,1)","ticklen":3.65296803652968,"tickwidth":0.66417600664176,"showticklabels":true,"tickfont":{"color":"rgba(77,77,77,1)","family":"","size":11.689497716895},"tickangle":-0,"showline":false,"linecolor":null,"linewidth":0,"showgrid":true,"gridcolor":"rgba(235,235,235,1)","gridwidth":0.66417600664176,"zeroline":false,"anchor":"x","title":{"text":"Probability","font":{"color":"rgba(0,0,0,1)","family":"","size":14.6118721461187}},"hoverformat":".2f"},"shapes":[{"type":"rect","fillcolor":"transparent","line":{"color":"rgba(51,51,51,1)","width":0.66417600664176,"linetype":"solid"},"yref":"paper","xref":"paper","x0":0,"x1":1,"y0":0,"y1":1}],"showlegend":false,"legend":{"bgcolor":"rgba(255,255,255,1)","bordercolor":"transparent","borderwidth":1.88976377952756,"font":{"color":"rgba(0,0,0,1)","family":"","size":11.689497716895}},"hovermode":"closest","barmode":"relative"},"config":{"doubleClick":"reset","showSendToCloud":false},"source":"A","attrs":{"65086cb1089b":{"x":{},"y":{},"label":{},"type":"bar"}},"cur_data":"65086cb1089b","visdat":{"65086cb1089b":["function (y) ","x"]},"highlight":{"on":"plotly_click","persistent":false,"dynamic":false,"selectize":false,"opacityDim":0.2,"selected":{"opacity":1},"debounce":0},"shinyEvents":["plotly_hover","plotly_click","plotly_selected","plotly_relayout","plotly_brushed","plotly_brushing","plotly_clickannotation","plotly_doubleclick","plotly_deselect","plotly_afterplot"],"base_url":"https://plot.ly"},"evals":[],"jsHooks":[]}</script>
</center>

---

The probability we want to find is the red area below. We will do this in SAS in just a second. The result is 0.02849.

<center>
<div id="htmlwidget-3a98ddbae2bf648b9222" style="width:600px;height:450px;" class="plotly html-widget"></div>
<script type="application/json" data-for="htmlwidget-3a98ddbae2bf648b9222">{"x":{"data":[{"orientation":"v","width":[1,1,1,1,1,1,1,1,1,1],"base":[0,0,0,0,0,0,0,0,0,0],"x":[0,1,2,3,4,5,6,7,8,9],"y":[0.0014108314823815,0.0101579866731468,0.0355529533560137,0.0805866942736309,0.132968045551491,0.170199098305909,0.175872401582772,0.150747772785233,0.109292135269294,0.0680039952786718],"text":["x: 0 Probability: 0.00141","x: 1 Probability: 0.01016","x: 2 Probability: 0.03555","x: 3 Probability: 0.08059","x: 4 Probability: 0.13297","x: 5 Probability: 0.17020","x: 6 Probability: 0.17587","x: 7 Probability: 0.15075","x: 8 Probability: 0.10929","x: 9 Probability: 0.06800"],"type":"bar","marker":{"autocolorscale":false,"color":"rgba(89,89,89,1)","line":{"width":1.88976377952756,"color":"transparent"}},"name":"Y < 10","legendgroup":"Y < 10","showlegend":true,"xaxis":"x","yaxis":"y","hoverinfo":"text","frame":null},{"orientation":"v","width":[1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1],"base":[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],"x":[10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36],"y":[0.0367221574504828,0.0173595653402283,0.0072331522250951,0.00267070236003511,0.000877516489725824,0.000257404836986242,6.75687697088885e-05,1.58985340491502e-05,3.35635718815394e-06,6.3594136196601e-07,1.08110031534221e-07,1.64739095671194e-08,2.24644221369811e-09,2.73479921667595e-10,2.96269915139895e-11,2.84419118534297e-12,2.40662331067484e-13,1.78268393383322e-14,1.14601110032136e-15,6.32281986384201e-17,2.95064926979291e-18,1.14218681411339e-19,3.56933379410432e-21,8.65293040994988e-23,1.52698771940293e-24,1.74512882217479e-26,9.69516012319318e-29],"text":["x: 10 Probability: 0.03672","x: 11 Probability: 0.01736","x: 12 Probability: 0.00723","x: 13 Probability: 0.00267","x: 14 Probability: 0.00088","x: 15 Probability: 0.00026","x: 16 Probability: 0.00007","x: 17 Probability: 0.00002","x: 18 Probability: 0.00000","x: 19 Probability: 0.00000","x: 20 Probability: 0.00000","x: 21 Probability: 0.00000","x: 22 Probability: 0.00000","x: 23 Probability: 0.00000","x: 24 Probability: 0.00000","x: 25 Probability: 0.00000","x: 26 Probability: 0.00000","x: 27 Probability: 0.00000","x: 28 Probability: 0.00000","x: 29 Probability: 0.00000","x: 30 Probability: 0.00000","x: 31 Probability: 0.00000","x: 32 Probability: 0.00000","x: 33 Probability: 0.00000","x: 34 Probability: 0.00000","x: 35 Probability: 0.00000","x: 36 Probability: 0.00000"],"type":"bar","marker":{"autocolorscale":false,"color":"rgba(255,0,0,1)","line":{"width":1.88976377952756,"color":"transparent"}},"name":"Y ≥ 10","legendgroup":"Y ≥ 10","showlegend":true,"xaxis":"x","yaxis":"y","hoverinfo":"text","frame":null},{"x":[9.5,9.5],"y":[0,0.193459641741049],"text":"","type":"scatter","mode":"lines","line":{"width":1.88976377952756,"color":"rgba(0,0,0,1)","dash":"dash"},"hoveron":"points","showlegend":false,"xaxis":"x","yaxis":"y","hoverinfo":"text","frame":null}],"layout":{"margin":{"t":40.8401826484018,"r":7.30593607305936,"b":37.2602739726027,"l":48.9497716894977},"plot_bgcolor":"rgba(255,255,255,1)","paper_bgcolor":"rgba(255,255,255,1)","font":{"color":"rgba(0,0,0,1)","family":"","size":14.6118721461187},"title":{"text":"Binomial(n = 36, p = 1/6)","font":{"color":"rgba(0,0,0,1)","family":"","size":17.5342465753425},"x":0,"xref":"paper"},"xaxis":{"domain":[0,1],"automargin":true,"type":"linear","autorange":false,"range":[-2.35,38.35],"tickmode":"array","ticktext":["0","10","20","30"],"tickvals":[0,10,20,30],"categoryorder":"array","categoryarray":["0","10","20","30"],"nticks":null,"ticks":"outside","tickcolor":"rgba(51,51,51,1)","ticklen":3.65296803652968,"tickwidth":0.66417600664176,"showticklabels":true,"tickfont":{"color":"rgba(77,77,77,1)","family":"","size":11.689497716895},"tickangle":-0,"showline":false,"linecolor":null,"linewidth":0,"showgrid":true,"gridcolor":"rgba(235,235,235,1)","gridwidth":0.66417600664176,"zeroline":false,"anchor":"y","title":{"text":"Value of Y","font":{"color":"rgba(0,0,0,1)","family":"","size":14.6118721461187}},"hoverformat":".2f"},"yaxis":{"domain":[0,1],"automargin":true,"type":"linear","autorange":false,"range":[0,0.193459641741049],"tickmode":"array","ticktext":["0.00","0.05","0.10","0.15"],"tickvals":[0,0.05,0.1,0.15],"categoryorder":"array","categoryarray":["0.00","0.05","0.10","0.15"],"nticks":null,"ticks":"outside","tickcolor":"rgba(51,51,51,1)","ticklen":3.65296803652968,"tickwidth":0.66417600664176,"showticklabels":true,"tickfont":{"color":"rgba(77,77,77,1)","family":"","size":11.689497716895},"tickangle":-0,"showline":false,"linecolor":null,"linewidth":0,"showgrid":true,"gridcolor":"rgba(235,235,235,1)","gridwidth":0.66417600664176,"zeroline":false,"anchor":"x","title":{"text":"Probability","font":{"color":"rgba(0,0,0,1)","family":"","size":14.6118721461187}},"hoverformat":".2f"},"shapes":[{"type":"rect","fillcolor":"transparent","line":{"color":"rgba(51,51,51,1)","width":0.66417600664176,"linetype":"solid"},"yref":"paper","xref":"paper","x0":0,"x1":1,"y0":0,"y1":1}],"showlegend":false,"legend":{"bgcolor":"rgba(255,255,255,1)","bordercolor":"transparent","borderwidth":1.88976377952756,"font":{"color":"rgba(0,0,0,1)","family":"","size":11.689497716895}},"hovermode":"closest","barmode":"relative"},"config":{"doubleClick":"reset","showSendToCloud":false},"source":"A","attrs":{"65081f646bc8":{"fill":{},"x":{},"y":{},"label":{},"type":"bar"},"650879e02791":{"xintercept":{}}},"cur_data":"65081f646bc8","visdat":{"65081f646bc8":["function (y) ","x"],"650879e02791":["function (y) ","x"]},"highlight":{"on":"plotly_click","persistent":false,"dynamic":false,"selectize":false,"opacityDim":0.2,"selected":{"opacity":1},"debounce":0},"shinyEvents":["plotly_hover","plotly_click","plotly_selected","plotly_relayout","plotly_brushed","plotly_brushing","plotly_clickannotation","plotly_doubleclick","plotly_deselect","plotly_afterplot"],"base_url":"https://plot.ly"},"evals":[],"jsHooks":[]}</script>
</center>

---

This means that *IF* the true probability of rolling a six with these dice is indeed `\(\frac{1}{6}\)`, the probability of rolling `\(10\)` or more sixes is `\(0.02849\)`.

Is this enough to convince you that the true probability is *NOT* `\(\frac{1}{6}\)`?

---

.center[
  **In SAS.**
]

---